Using external memory devices to improve system performance

ABSTRACT

The invention is directed towards a system and method that utilizes external memory devices to cache sectors from a rotating storage device (e.g., a hard drive) to improve system performance. When an external memory device (EMD) is plugged into the computing device or onto a network in which the computing device is connected, the system recognizes the EMD and populates the EMD with disk sectors. The system routes I/O read requests directed to the disk sector to the EMD cache instead of the actual disk sector. The use of EMDs increases performance and productivity on the computing device systems for a fraction of the cost of adding memory to the computing device.

RELATED APPLICATION(S)

This application is a Continuation of and claims benefit from U.S.patent application Ser. No. 13/187,757 that was filed on Jul. 21, 2011,and that is a Continuation of U.S. Pat. No. 8,006,037 that was issued onAug. 23, 2011, and that is a Continuation of U.S. Pat. No. 7,805,571that was issued on Sep. 9, 2010, and that is a Continuation of U.S. Pat.No. 7,490,197 that was issued on Feb. 10, 2009, each of which isincorporated herein by reference in its entirety.

FIELD OF THE INVENTION

This invention relates generally to computer systems and, moreparticularly, relates to improving performance of computer systems.

BACKGROUND OF THE INVENTION

Computing devices such as personal computers, game consoles, smartphones, and the like often utilize a time-consuming process in order toload and cache pages used by applications into memory. The pages aretypically stored on a rotating non-volatile media such as a magnetichard disk (e.g., a hard drive). However, the device's processor executesinstructions only from addressable memory such as DRAM or some othertype of volatile electronic memory. The operating systems used in thecomputing devices cache the pages used by applications in memory so thatthe applications do not need to load pages from the rotating media asfrequently.

The transfer of the pages from the hard drive is slow, particularly whenthe application is loading a large file. This is also prevalent inrestoring the computer system from hibernate mode. A significant factorin the transfer time is due to the disk drive spin up speed. Arelatively small disk spinning at a relatively slow RPM requires 5 to 6seconds to spin up and be usable. Larger disks such as multi-platterdevices and those spinning at faster RPMs require 10 to 12 seconds ormore to spin up.

This problem gets worse as applications grow in size to incorporatesecurity fixes and become more reliable. These applications oftenrequire more memory to operate without having to continually transferdata to and from the rotating storage media. However, upgrading thememory of machines is often too costly to undertake for corporations andend users or is beyond the skill level of individual users. Although thecost of memory itself is low, the labor and downtime involved inphysically opening each machine and adding RAM may cost several hundreddollars.

Another problem where upgrading the memory of machines is often toocostly to undertake is when a system is required to occasionally executelarger and more complex applications than normal. For example, anaccounting staff of a company might need to run consolidationapplications a few times a month. The larger and more complexapplications require more memory to operate efficiently. Although thecost of memory itself is low, the labor and downtime involved inphysically opening each machine and adding RAM may cost several hundreddollars. This cost may not justify the additional memory for the fewtimes the application is run.

BRIEF SUMMARY OF THE INVENTION

The invention is directed towards an improved memory managementarchitecture that provides a system, method, and mechanism that utilizesexternal memory (volatile or non-volatile) devices to cache sectors fromthe hard disk (i.e., disk sectors) and/or slower memory components toimprove system performance. When an external memory device (EMD) isplugged into the computing device or onto a network in which thecomputing device is connected, the system recognizes the EMD andpopulates the EMD with disk sectors and/or memory sectors. The systemroutes I/O read requests directed to the sector to the EMD cache insteadof the actual sector. If the EMD is connected to the USB2 local bus, theaccess time can be twenty times faster that reading from the hard disk.The use of EMDs increases performance and productivity on the computingdevice systems for a fraction of the cost of adding memory to thecomputing device. Additionally, consumer devices such as Xbox® can runricher software with the memory of EMDs.

The system detects when an EMD is first used with respect to thecomputing device. The type of EMD is detected and a driver is installedthat is used to cache disk sectors on the EMD. The driver uses the EMDas an asynchronous cache, caching sectors from any disk and/or slowermemory device on the system. If no prior knowledge of which sectors arevaluable in terms of frequent access, the system may use data on thecomputing machine to determine which sectors are used to populate theEMD cache. Alternatively, the system populates the EMD cache with aparticular sector when that particular sector is accessed duringoperation. The next time that particular sector is to be accessed for aread operation, the system directs the read operation to access the copyfrom the EMD.

The system may track usage patterns and determine which disk sectors aremost frequently accessed. On subsequent uses of the EMD, the systemcaches those sectors that are most frequently accessed onto the EMD. Ifthe EMD is present when the computing device is powered up, the EMD canbe pre-populated with data during start-up of the operating system.

Additional features and advantages of the invention will be madeapparent from the following detailed description of illustrativeembodiments which proceeds with reference to the accompanying figures.

BRIEF DESCRIPTION OF THE DRAWINGS

While the appended claims set forth the features of the presentinvention with particularity, the invention, together with its objectsand advantages, may be best understood from the following detaileddescription taken in conjunction with the accompanying drawings ofwhich:

FIG. 1 is a block diagram generally illustrating an exemplary computersystem on which the present invention resides;

FIG. 2 is a block diagram representing a memory management architecturein accordance with an aspect of the invention; and

FIGS. 3a-3b are a flow chart generally illustrating the steps theinvention takes in utilizing external memory devices to improve systemperformance.

DETAILED DESCRIPTION OF THE INVENTION

The invention is directed towards an improved memory managementarchitecture that provides a system, method, and mechanism that utilizesexternal memory (volatile or non-volatile) devices to cache sectors fromthe hard disk (i.e., disk sectors) or from slower memory devices toimprove system performance. For example, many classes of portablecomputing devices have no hard drives or rotating media storage devices,but still implement hierarchical memory architectures. These portablecomputing devices would benefit greatly from this invention in that itwould allow them to execute larger and more complex enterpriseapplications within the office place. With the advent of 802.11n,200-500 Mb wireless connectivity will be available to any wirelessdevice and the use of external memory devices and/or network basedmemory servers will improve system performance.

The external memory is used to cache data from devices that aregenerally slower in terms of accessing data such that access times fordata used by applications/operating systems can be accessed quicker,thereby improving performance. For older computing devices in whichadding actual RAM is too costly, the use of external memory devices willincrease performance and productivity on the older devices for afraction of the cost and enable users to reap the reliability, security,and productivity improvements of newer software applications on existinghardware. For example, consumer devices such as Xbox® benefit by runningricher software in terms of improved graphics and performance.Additionally, the amount of memory required for this purpose is likelymuch less than the amount of memory required to update a system up to agiven level.

Turning to the drawings, wherein like reference numerals refer to likeelements, the invention is illustrated as being implemented in asuitable computing environment. Although not required, the inventionwill be described in the general context of computer-executableinstructions, such as program modules, being executed by a personalcomputer. Generally, program modules include routines, programs,objects, components, data structures, etc. that perform particular tasksor implement particular abstract data types. Moreover, those skilled inthe art will appreciate that the invention may be practiced with othercomputer system configurations, including hand-held devices,multi-processor systems, microprocessor based or programmable consumerelectronics, network PCs, minicomputers, mainframe computers, and thelike. The invention may also be practiced in distributed computingenvironments where tasks are performed by remote processing devices thatare linked through a communications network. In a distributed computingenvironment, program modules may be located in both local and remotememory storage devices.

FIG. 1 illustrates an example of a suitable computing system environment100 on which the invention may be implemented. The computing systemenvironment 100 is only one example of a suitable computing environmentand is not intended to suggest any limitation as to the scope of use orfunctionality of the invention. Neither should the computing environment100 be interpreted as having any dependency or requirement relating toany one or combination of components illustrated in the exemplaryoperating environment 100.

The invention is operational with numerous other general purpose orspecial purpose computing system environments or configurations.Examples of well known computing systems, environments, and/orconfigurations that may be suitable for use with the invention include,but are not limited to: personal computers, server computers, hand-heldor laptop devices, tablet devices, multiprocessor systems,microprocessor-based systems, set top boxes, programmable consumerelectronics, network PCs, game consoles, smart phones, personal dataassistants, minicomputers, mainframe computers, distributed computingenvironments that include any of the above systems or devices, and thelike.

The invention may be described in the general context ofcomputer-executable instructions, such as program modules, beingexecuted by a computer. Generally, program modules include routines,programs, objects, components, data structures, etc. that performparticular tasks or implement particular abstract data types. Theinvention may also be practiced in distributed computing environmentswhere tasks are performed by remote processing devices that are linkedthrough a communications network. In a distributed computingenvironment, program modules may be located in local and/or remotecomputer storage media including memory storage devices.

With reference to FIG. 1, an exemplary system for implementing theinvention includes a general purpose computing device in the form of acomputer 110. Components of computer 110 may include, but are notlimited to, a processing unit 120, a system memory 130, and a system bus121 that couples various system components including the system memoryto the processing unit 120. The system bus 121 may be any of severaltypes of bus structures including a memory bus or memory controller, aperipheral bus, and a local bus using any of a variety of busarchitectures. By way of example, and not limitation, such architecturesinclude Industry Standard Architecture (ISA) bus, Micro ChannelArchitecture (MCA) bus, Enhanced ISA (EISA) bus, Video ElectronicsStandards Association (VESA) local bus, Universal Serial Bus (USB), andPeripheral Component Interconnect (PCI) bus also known as Mezzanine bus.

Computer 110 typically includes a variety of computer readable media.Computer readable media can be any available media that can be accessedby computer 110 and includes both volatile and nonvolatile media, andremovable and non-removable media. By way of example, and notlimitation, computer readable media may comprise computer storage mediaand communication media. Computer storage media includes volatile andnonvolatile, removable and non-removable media implemented in any methodor technology for storage of information such as computer readableinstructions, data structures, program modules or other data. Computerstorage media includes, but is not limited to, RAM, ROM, EEPROM, flashmemory or other memory technology, CD-ROM, digital versatile disks (DVD)or other optical disk storage, magnetic cassettes, magnetic tape,magnetic disk storage or other magnetic storage devices, or any othermedium which can be used to store the desired information and which canbe accessed by computer 110. Communication media typically embodiescomputer readable instructions, data structures, program modules orother data in a modulated data signal such as a carrier wave or othertransport mechanism and includes any information delivery media. Theterm “modulated data signal” means a signal that has one or more of itscharacteristics set or changed in such a manner as to encode informationin the signal. By way of example, and not limitation, communicationmedia includes wired media such as a wired network or direct-wiredconnection, and wireless media such as acoustic, RF, infrared and otherwireless media. The term “computer storage media” as used herein refersto an article of manufacture that is not a signal or carrier wave perse. Combinations of the any of the above should also be included withinthe scope of computer readable media.

The system memory 130 includes computer storage media in the form ofvolatile and/or nonvolatile memory such as read only memory (ROM) 131and random access memory (RAM) 132. A basic input/output system 133(BIOS), containing the basic routines that help to transfer informationbetween elements within computer 110, such as during start-up, istypically stored in ROM 131. RAM 132 typically contains data and/orprogram modules that are immediately accessible to and/or presentlybeing operated on by processing unit 120. By way of example, and notlimitation, FIG. 1 illustrates operating system 134, applicationprograms 135, other program modules 136, and program data 137.

The computer 110 may also include other removable/non-removable,volatile/nonvolatile computer storage media. By way of example only,FIG. 1 illustrates a hard disk drive 141 that reads from or writes tonon-removable, nonvolatile magnetic media, a magnetic disk drive 151that reads from or writes to a removable, nonvolatile magnetic disk 152,and an optical disk drive 155 that reads from or writes to a removable,nonvolatile optical disk 156 such as a CD ROM or other optical media.Other removable/non-removable, volatile/nonvolatile computer storagemedia that can be used in the exemplary operating environment include,but are not limited to, magnetic tape cassettes, flash memory cards,digital versatile disks, digital video tape, solid state RAM, solidstate ROM, and the like. The hard disk drive 141 is typically connectedto the system bus 121 through a non-removable memory interface such asinterface 140, and magnetic disk drive 151 and optical disk drive 155are typically connected to the system bus 121 by a removable memoryinterface, such as interface 150.

The drives and their associated computer storage media, discussed aboveand illustrated in FIG. 1, provide storage of computer readableinstructions, data structures, program modules and other data (e.g.,multimedia data, audio data, video data, etc.) for the computer 110. InFIG. 1, for example, hard disk drive 141 is illustrated as storingoperating system 144, application programs 145, other program modules146, and program data 147. Note that these components can either be thesame as or different from operating system 134, application programs135, other program modules 136, and program data 137. Operating system144, application programs 145, other program modules 146, and programdata 147 are given different numbers hereto illustrate that, at aminimum, they are different copies. A user may enter commands andinformation into the computer 110 through input devices such as akeyboard 162, a pointing device 161, commonly referred to as a mouse,trackball or touch pad, a microphone 163, and a tablet or electronicdigitizer 164. Other input devices (not shown) may include a joystick,game pad, satellite dish, scanner, or the like. These and other inputdevices are often connected to the processing unit 120 through a userinput interface 160 that is coupled to the system bus, but may beconnected by other interface and bus structures, such as a parallelport, game port or a universal serial bus (USB). A monitor 191 or othertype of display device is also connected to the system bus 121 via aninterface, such as a video interface 190. The monitor 191 may also beintegrated with a touch-screen panel or the like. Note that the monitorand/or touch screen panel can be physically coupled to a housing inwhich the computing device 110 is incorporated, such as in a tablet-typepersonal computer. In addition, computers such as the computing device110 may also include other peripheral output devices such as speakers197 and printer 196, which may be connected through an output peripheralinterface 194 or the like.

The computer 110 may operate in a networked environment using logicalconnections to one or more remote computers, such as a remote computer180. The remote computer 180 may be a personal computer, a server, arouter, a network PC, a peer device or other common network node, andtypically includes many or all of the elements described above relativeto the computer 110, although only a memory storage device 181 has beenillustrated in FIG. 1. The logical connections depicted in FIG. 1include a local area network (LAN) 171 and a wide area network (WAN)173, but may also include other networks. Such networking environmentsare commonplace in offices, enterprise-wide computer networks, intranetsand the Internet. For example, the computer system 110 may comprise thesource machine from which data is being migrated, and the remotecomputer 180 may comprise the destination machine. Note however thatsource and destination machines need not be connected by a network orany other means, but instead, data may be migrated via any media capableof being written by the source platform and read by the destinationplatform or platforms.

When used in a LAN networking environment, the computer 110 is connectedto the LAN 171 through a network interface or adapter 170. When used ina WAN networking environment, the computer 110 typically includes amodem 172 or other means for establishing communications over the WAN173, such as the Internet. The modem 172, which may be internal orexternal, may be connected to the system bus 121 via the user inputinterface 160, or other appropriate mechanism. In a networkedenvironment, program modules depicted relative to the computer 110, orportions thereof, may be stored in the remote memory storage device. Byway of example, and not limitation, FIG. 1 illustrates remoteapplication programs 185 as residing on memory device 181. It will beappreciated that the network connections shown are exemplary and othermeans of establishing a communications link between the computers may beused.

In the description that follows, the invention will be described withreference to acts and symbolic representations of operations that areperformed by one or more computers, unless indicated otherwise. As such,it will be understood that such acts and operations, which are at timesreferred to as being computer-executed, include the manipulation by theprocessing unit of the computer of electrical signals representing datain a structured form. This manipulation transforms the data or maintainsit at locations in the memory system of the computer, which reconfiguresor otherwise alters the operation of the computer in a manner wellunderstood by those skilled in the art. The data structures where datais maintained are physical locations of the memory that have particularproperties defined by the format of the data. However, while theinvention is being described in the foregoing context, it is not meantto be limiting as those of skill in the art will appreciate that variousof the acts and operation described hereinafter may also be implementedin hardware.

Turning now to FIG. 2, the present invention provides a memory manager200 controlling conventional device memory 202 and is in communicationwith external memory device (EMD) manager 204. The EMD manager 204 isunder the memory manager 200 and above the physical hardware 206 ₁, 206₂, 208 and network 210. The physical hardware may be a hard drive, amultimedia drive such as a CD drive, a DVD drive, or a combinationCD/DVD drive, an optical disk, etc. located locally or remotelyaccessible via the network. While EMD manager 204 is shown separately,it is recognized that the EMD manager 204 may be integrated with memorymanager 200. EMD manager 204 detects when an external memory device(EMD) 212 is accessible via conventional methods such as plug-n-play andthe like. The EMD 212 may be in the form of a removable solid statenon-volatile memory device which can be plugged into the computingdevice, such as one according to the CompactFlash specification asmaintained by the CompactFlash Association, or the like. It may also bein the form of a volatile memory device. The EMD can in fact be housedwithin existing externally attached products, such as a mouse, akeyboard, or a network attached device and there can be multiple suchdevices attached at a time. Another alternative location of the externalmemory device is at a remote location on network 210 or part of thenetwork infrastructure such as memory on a server.

The present invention leverages the memory available for use in the EMDto maintain in memory the disk sectors that are likely to be used byapplications and directs I/O requests that are directed to data that isin disk sectors copied into the EMD memory to be read from the EMDmemory instead of the sector on disk.

With reference to FIGS. 3a and 3b , the steps the invention performs toutilize external memory devices shall now be described. In thedescription that follows, the sectors used to describe the inventionwill reside on a hard drive 206. While the invention is being describedin the foregoing context, it is not meant to be limiting as those ofskill in the art will appreciate that disk sectors from other devicesthat require spin-up such as CD/DVD device 208 and the like may becached on disk. The sectors that are cached may also reside on slowermemory devices. While FIGS. 3a and 3b show steps serially, it should beunderstood that the steps may be taken in different order and/or inparallel. EMD manager 204 detects when an EMD 212 is available (step300). One approach to detect an EMD is the detection interface describedin U.S. patent application Ser. No. 10/837,986, filed May 3, 2004,entitled “Non-Volatile Memory Cache Performance Improvement”, herebyincorporated by reference in its entirety. Other methods may be usedsuch as conventional plug and play methods. The size and type of memoryavailable in the EMD 212 is determined. If the EMD 212 is being used forthe first time in the computing device, a driver for the EMD 212 isinstalled (step 302). The driver is used to communicate with the EMD 212and uses the EMD as an asynchronous block cache to cache sectors fromdisks 206 on the system. The updating of the cache is asynchronous inthe event that the EMD may be slow and waiting for it to be updated canresult in increased latency for the original read request.

If other EMDs are available for use, the system prioritizes how the EMDSwill be populated by caching disk sectors that are more likely to beused on EMDs that have better bandwidth and latency in comparison toother available EMDs (step 304). Some computing devices keep track ofdisk usage such as which disk sectors are most frequently accessed bythe operating system and by applications, last access times, accesspatterns, access frequency, and the like. If this history is available,the EMD is populated based on the history (step 306). If the history isnot available, the EMD is populated with the disk sectors being accessedby the applications (or computing device) during the time theapplication is reading from disk (step 308). Note that the EMD may bepopulated in the format required by the EMD. The usage information(i.e., history) of disk sectors is tracked to determine which sectorsshould be mirrored onto the EMD the next time the EMD is available foruse. The algorithms used are similar to the algorithms used toproactively manage page memory as described in U.S. patent applicationSer. No. 10/325,591, filed Dec. 20, 2002, entitled “Methods andMechanisms for Proactive Memory Management,” which is herebyincorporated by reference in its entirety. The difference is thatinstead of determining which pages in memory are useful to cache, thepresent invention determines which disk sectors are useful to cache.

In one embodiment wherein the computing device is in a networked system,a network server retains information about the computing device andemploys remote algorithms that assist the EMD manager 204 in themanagement of local memory for the computing device. This embodiment isparticularly suitable for low-end clients that don't have the memory orcomputer power to determine which disk sectors should be cached. Theremote algorithms perform a detailed analysis on data patterns, accesspatterns, etc. on the client and produce more optimum results than thelow-end client could produce.

During operation, an application or the computing device may write to adisk sector that is copied to an EMD. The EMD is never written to by theapplication or computing device. Instead, the write operation is appliedto the disk sector. After the write operation is completed, the disksector is copied back onto the EMD (step 310). This approach is used sothat if the EMD is removed, no data is lost such as would be the case ina remote file system when the link to the remote file system is notoperable; instead, the computing device reads from disk instead of theEMD. As a result, the invention is more resistant to connectivity issuessuch as lost connections, removal of EMDs, etc.

Whenever an I/O read request is received, EMD manager 204 checks to seeif the request is directed to a disk sector that has been copied to thememory of an EMD 212. If the read request is directed to a disk sectorthat has been copied to the memory of an EMD, the EMD manager 204redirects the read request to the EMD (step 312). The result is that theread request is completed faster than if the read request was completedat the hard disk 206.

An EMD 212 can be removed by a user at any time. When an EMD is removed,the system detects the removal. If other EMDs are available, theremaining EMDs are repopulated (step 314) if the EMD that was removedwas not the slowest EMD available. If other EMDs are not available (orif the EMD that was removed was the slowest EMD), data is read from thehard disk (step 316). Steps 300 to 316 are repeated whenever an EMD isadded or removed and steps 310 and 312 are repeated for as long as anEMD is available for use.

Note that if the EMD is non-volatile, the EMD memory can bepre-populated with sectors having configuration data during power downor when hibernating. During power-up or restoration, the contents of theEMD can be read while the disk is spinning up. The use of this techniquecan decrease the boot time and the hibernate awaken time of a computersystem. Further details can be found in U.S. patent application Ser. No.10/186,164, filed Jun. 27, 2002, entitled “Apparatus and Method toDecrease Boot Time and Hibernate Awaken Time of a Computer System,”hereby incorporated by reference in its entirety.

Now that the overall steps have been described, the performanceimprovements shall be discussed. The key factors that determine theperformance improvements that can be expected from external memorydevices are the transfer latency and throughput for the EMD and its bus(e.g. USB1/2, PCMCIA, Ethernet 100BaseT, etc.), the size of the externalmemory, the policies used in managing the cache, and the scenarios andworkloads of how the external memory is used.

The transfer latency and throughput for the most typical busses EMD maybe plugged in varies. It is expected that the bus becomes the primarybottleneck for most operations if the EMD consists of regular RAMpackaged as a device that can be plugged into the particular bus. Thebus latency and throughput for USB1, USB2 and PCI/PCMCIA is estimated byissuing unbuffered disk I/Os of increasing sizes (4 KB, 8 KB, 16 KB, 32KB and 64 KB) that should hit the track buffer (which is typicallyregular memory) of the disk plugged into that bus. The following valuesof Table 1 were derived by simply fitting a line to the times it took totransfer the I/O sizes.

TABLE 1 Time to Total Time to Setup Transfer each Transfer Bus Type Time(us) KB after Setup (us) 4 KB (us) PCI/ 100 15 160 PCMCIA(Cardbus) USB 2400 30 520 USB 1 4000 1000 8000

In order to be meaningful as a disk cache, copying data from the EMDmust be faster than going to the disk for it. A 4 KB random disk I/Othat involves a seek takes anywhere from 5-15 ms on typical desktop andlaptop disks. Assume that it takes 10 ms for a 4 KB disk I/O with seek,data could have been retrieved 60× faster from an EMD cache on PCMCIA,or 20× faster from an EMD on USB2. Overall, USB2 seems to be a verysuitable bus for plugging in EMDs.

It should be noted that one issue with USB1 is that the 4 ms setup timeswould make any performance gains unlikely. This can be worked around byalways keeping an isochronous transfer channel open. Obtaining 4 KBsfrom an EMD on USB 1 would then be typically twice as fast thenobtaining it from a disk with a seek. Due to the low throughput rateover USB 1, it would still be faster to go to the disk for 16 KB, 32 KBand 64 KB I/Os that are typically seen on client systems. However, a USB1 cache used only for the pagefile and file system metadata which istypically accessed with 4 KB random I/Os can still deliver a performanceboost.

USB 2 adoption started only after service pack 1 of Windows XP® wasreleased. Most of the 64 MB and 128 MB systems that would benefit mostfrom EMD will not typically have USB 2. However, these systems usuallydo have a 100BaseT Ethernet network cards. Transfer times of 10 MB/swould be sufficient for significant performance gains from an EMD. AnEMD could be attached as a pass through network device per computer, orcould even be pushed into the network switches to improve theperformance of a small network of computers. Going beyond the switchintroduces many reliability and security issues due to shared networkbandwidth, but could be done.

As with any cache, the actual policies used in managing which data tokeep in the cache is a big factor in determining the resultingperformance gains. If an EMD is used as a block cache for underlyingdisks and other devices, the EMD cache can be populated when reads fromthe underlying device completes, as well as when writes are issued fromapplications and file systems. As previously described, the data in theEMD cache will need to be updated asynchronously in order to avoidincreasing the time of the original device requests. If a request comesfor a range that is being asynchronously updated, it can simply bepassed down to the underlying device. If the asynchronous update isoutstanding, there must have been a very recent request for the samerange that initiated the update, and the data for the range is likely tobe cached at the device (e.g. track buffer) or controller.

Typically block caches are managed with an LRU algorithm. In thealgorithm, the referenced blocks are put to the end of the LRU listwhenever a read request hits or misses the cache. When a block that isnot in the cache is read or written to, blocks from the front of the LRUlist are repurposed to cache the contents of the new blocks. As aresult, LRU algorithms are prone to erosion because valuable blocks inthe cache are churned through over time. Algorithms such as those thatbreak the list to multiple prioritized sub-lists and maintain richer usehistory beyond the last access time will be more resilient.

On Windows NT, caching of file and page data is done by the memorymanager via a standby page list. File systems, registry and other systemcomponents use the file object/mapping mechanisms to cache their data atthe same level through the memory and cache manager. If another cache isput at any other level, it results in double caching of the data. Thisholds true for EMD caches as well. In order to avoid this, the memorymanager of the present invention can be extended to push less valuablestandby list pages to the slower external memory devices. Whenever thosepages are accessed, the memory manager can allocate physical memorypages and copy the data back from the external memory device. The EMDmemory manager and an associated cache manager can use page priorityhints that U.S. patent application Ser. No. 10/325,591 provides for aproactive and resilient management of the unified cache of pages. Sincethis will require kernel memory manager changes, any EMD solutions builtfor Windows XP are likely to suffer from double caching of the data.Simulations show that in spite of the double caching, substantialperformance gains are still possible.

Another important parameter for caching is the block size and the amountof clustering and read-ahead. Whenever there is a miss in the cache,even if a smaller amount of data is requested, one needs to read atleast a block size of data from the underlying disk or device andpossibly even cluster more blocks around the requested offset.Clustering may eliminate future seeks back to the same position on thedisk. However, it may also increase the completion time of the originalrequest and even cause more chum in the LRU list as more blocks arereferenced for each request. Further, read ahead may be queued to geteven more consecutive data from the disk while it is efficient to do so,without impacting the time for the original request. However, this mayresult in increasing the latency for a subsequent request that needs toseek to somewhere else on the device.

It should be noted that the list of device locations that are deemedvaluable by the cache can be persisted across power transitions such asboot or even periods of intense use that purge the regular contents ofthe cache. This list can be used to repopulate the cache contents aftersuch a transition with proper prioritization support for background I/O.

As with any performance analysis, it is crucial to look atrepresentative scenarios and workloads to getting meaningful and usefuldata. In order to characterize the performance improvements that can beexpected from EMD caches on existing Windows (XP & 2000), experimentswith simple LRU write-through block caching at the disk level wereperformed. As discussed above, this will suffer from double caching ofthe data. However, these experiments are easier to emulate, simulate andactually build such EMD caches and measure their impact. Results showthat even such a simple cache can have a big impact on disk and systemperformance. Integration with the computing device's memory manager andusing a smarter policy would further increase the gains.

Since the experiment basically caches for the disk accesses, the successof the cache can be measured by comparing the overall time for theplayback of the same set of disk accesses that are captured from arepresentative workload or scenario, without the cache and with variousconfigurations of the cache. In most client scenarios, reductions indisk read times result in a proportional increase in responsiveness orbenchmark scores.

In order to determine the real world impact of an EMD cache, twoscenarios were looked at. One used disk traces captured from realend-user systems over hours on 128 MB and 256 MB systems. Another useddisk traces from industry benchmarks such as Business Winstone 2001.Content Creation Winstone 2002, and a modified version of BusinessWinstone that uses Office 2003 applications. Traces were obtained atmultiple memory sizes, so the gains could be compared from a simple EMDcache to actually increasing the system memory size.

EMD devices can be accurately emulated by using a regular block cacheand adding a delay to cache hits based on the desired EMD bus. Aftercopying the requested bytes from memory, one can determine the transfertime that is calculated for the desired EMD bus based on the setup timeand throughput values such as the ones in Table 1.

The procedure for this evaluation is to: configure the target system torun at the target memory size with /maxmem boot.ini switch; run thetypical use scenario or an industry benchmark and trace the generateddisk I/Os; configure the block cache with the desired parameters for thecache size and throughput/latency for the EMD device; replay the traceddisk I/Os and capture the resulting disk I/Os due to cache misses; andcompare the times and disk accesses for the two runs.

Ideally the scenarios should be run with the appropriately configuredblock cache and the end results (response times or benchmark scores)compared. However, if the link between disk times and the end results isalready established, simply playing back the captured disk I/Os consumeless time for the numerous EMD configurations that need to be evaluated.A simple simulator was used to roughly estimate the potential gains froman EMD cache. This allowed the processing of hours-long disk traces from128 MB customer systems as well as from internal development systems andmeasure the impact of various configurations of EMD caches. In order tosimplify things further, we focused on the time it took the disk toprocess the reads and ignored the disk write times. Representative seektimes were determined by ignoring seek times smaller than 2 ms andlarger than 20 ms. The last couple positions of the disk head weretracked to simulate “track buffering.” In spite of the complicationsabove, the disk simulation is typically within an acceptable range: 75%of the predictions are within 15% of the actual times. Any mispredictionis typically due to the conservative simulation and prediction of higherdisk read times. Even though the disk simulator may not alwaysaccurately capture the performance characteristics of a disk in aspecific trace, its own performance characteristics are representativeand typical of an actual desktop/laptop disk.

Table 2 shows the reduction in disk read times in EMD cache simulationof disk traces that were acquired during actual use of various computingsystems over hours of operation.

TABLE 2 Gains from EMD cache for actual end-user use of systemsSimulated Disk Read Time % with a USB2 Simulated Disk EMD Cache of SizeSystem Read Time (sec) 0 MB 32 MB 64 MB 128 MB 256 MB 512 MB System 1(128 MB) 1259 100% 89% 70% 37% 18% 18% System 2 (128 MB) 1011 100% 90%70% 38% 22% 22% System 3 (128 MB) 2158 100% 88% 72% 44% 25% 20% System 4(128 MB) 866 100% 90% 80% 63% 48% 37% System 5 (256 MB) 1747 100% 92%85% 70% 52% 40% System 6 (256 MB) 2187 100% 94% 87% 76% 66% 57%As an example of how to interpret data from Table 2, consider system 1:a 128 MB USB2 EMD device will result in 37% of the disk read time thatthe current user is experiencing (i.e., a 63% reduction).

Systems 1 and 2 are from a corporation that wanted to upgrade to WindowsXP, Office 2003 and latest SMS on their 128 MB systems, but hitsignificant slowdowns when running their line of business software. Thesystem 3 trace is from a laptop. It can be seen that the largestimprovements in these systems are systems with slower disks and only 128MB of memory.

The bottom three systems (systems 4, 5, and 6) are developer systems onwhich heavy weight development tasks including building, syncing &processing of large files were performed. These systems have fasterdisks and the most disk I/Os generated by these tasks are sequential anddo not benefit from a simple LRU block cache as much because they do notre-access the same sectors on the disk many times (e.g. syncing). Thusthe overall disk time is not as representative of the end userresponsiveness. The cache may have reduced the time for UI blocking diskreads significantly.

Table 3 shows the reduction in disk read times in EMD cache simulationof disk traces that were acquired during Content Creation Winstone 2002.

TABLE 3 Gains from EMD cache for Content Creation Winstone 2002Simulated Disk Read Time % with a USB2 EMD Simulated Disk Cache of SizeSystem Read Time (s) 0 MB 32 MB 64 MB 128 MB 256 MB 512 MB Laptop150(128MB) 241 100% 88% 76% 62% 46% 39% Laptop154(128 MB) 172 100% 89% 76% 63%46% 40% Desktop100(128 MB) 173 100% 90% 78% 65% 46% 40% Desktop949(128MB) 142 100% 89% 79% 67% 48% 42% Laptop150(256 MB) 64 100% 93% 86% 72%55% 54% Laptop154(256 MB) 55 100% 90% 84% 70% 56% 56% Desktop100(256 MB)47 100% 95% 87% 76% 60% 59% Desktop949(256 MB) 34 100% 94% 88% 80% 70%70%

Table 4 shows the reduction in disk read times in EMD cache simulationof disk traces that were acquired during Business Winstone 2001.

TABLE 4 Gains from EMD cache for Business Winstone 2001 Simulated DiskRead Time % with a USB2 Simulated Disk EMD Cache of Size System ReadTime (s) 0 MB 32 MB 64 MB 128 MB 256 MB 512 MB Laptop150(128 MB) 176100% 84% 75% 60% 41% 37% Laptop159(128 MB) 226 100% 88% 76% 60% 42% 37%Desktop094(128 MB) 90 100% 90% 83% 71% 54% 52% Desktop211(128 MB) 83100% 91% 84% 72% 59% 57% Laptop150 (256 MB) 93 100% 82% 79% 67% 56% 55%Laptop159(256 MB) 76 100% 87% 86% 76% 69% 69% Desktop211(256 MB) 40 100%94% 92% 85% 79% 78% Desktop094(256 MB) 40 100% 95% 93% 85% 80% 79%As in previous cases, the improvements seen on systems with 128 MB andslower disks are the largest. Business Winstone 2001 starts to mostlyfit in memory in 256 MBs, so the overall disk times and the gains fromEMD are smaller in this system memory size.

Table 5 compares the gains from adding EMD cache to a system to actuallyadding more physical memory when running Content Creation Winstone 2002.As previously noted, the EMD cache simulation suffers from doublecaching of the data and is managed with a simple LRU policy. Typicallyadding more physical memory to the system will deliver betterperformance in a bigger number of scenarios. On the other hand, if theEMD cache can be integrated with the memory manager and managed with thesame advanced algorithms that U.S. patent application Ser. No.10/325,591 can provide, it can deliver performance gains comparable toadding actual memory to the system.

TABLE 5 Comparison of gains from USB2 EMD cache and actual increase insystem memory Simulated Disk Read Time (s) with USB2 EMD Cache of SizeSystem & Memory Size 0 MB 32 MB 64 MB 128 MB 256 MB 512 MB Laptop150(128MB) 266 212 184 149 110 93 Laptop150(256 MB) 76 60 56 46 36 35Laptop150(512 MB) 27 24 23 21 21 20

From the foregoing, it can be seen that a system and method to improvethe performance of a computing device using external memory has beendescribed. The invention allows legacy computing devices and otherdevices with low amounts of memory to effectively upgrade the memorywithout having to physically open the device. Productivity gains interms of faster and more reliable performance can be achieved using theexternal memory. Sectors from rotating storage media and slower memorydevices are asynchronously cached in the external memory. Unlike remotefile systems, data is not lost if the external memory is removed as thedata is still on the rotating storage media or slower memory devices.

All of the references cited herein, including patents, patentapplications, and publications, are hereby incorporated in theirentireties by reference. The use of the terms “a” and “an” and “the” andsimilar referents in the context of describing the invention (especiallyin the context of the following claims) is to be construed to cover boththe singular and the plural, unless otherwise indicated herein orclearly contradicted by context. The terms “comprising,” “having,”“including,” and “containing” are to be construed as open-ended terms(i.e., meaning “including, but not limited to,”) unless otherwise noted.All methods described herein can be performed in any suitable orderunless otherwise indicated herein or otherwise clearly contradicted bycontext. The use of any and all examples, or exemplary language (e.g.,“such as”) provided herein, is intended merely to better illuminate theinvention and does not pose a limitation on the scope of the inventionunless otherwise claimed. For example, the Windows® operating system wasreferenced to describe the invention. Those skilled in the art willrecognize that the invention may be implemented on other operatingsystems such as Linux, SunOs, and the like. No language in thespecification should be construed as indicating any non-claimed elementas essential to the practice of the invention.

In view of the many possible embodiments to which the principles of thisinvention may be applied, it should be recognized that the embodimentdescribed herein with respect to the drawing figures is meant to beillustrative only and should not be taken as limiting the scope ofinvention. For example, those of skill in the art will recognize thatthe elements of the illustrated embodiment shown in software may beimplemented in hardware and vice versa or that the illustratedembodiment can be modified in arrangement and detail without departingfrom the spirit of the invention. Therefore, the invention as describedherein contemplates all such embodiments as may come within the scope ofthe following claims and equivalents thereof.

We claim:
 1. A method performed on a computing device that comprises aprocessor and memory, the method comprising: determining, by thecomputing device in response to a read request for data where the readrequest is directed to a hard disk, that the data is available from aremovable solid-state memory device that is separate from the hard disk,where the separate removable solid-state memory device is configured forcompleting the read request faster than the hard disk; and redirecting,by the computing device in response to the determining, the read requestfrom the hard disk to the separate removable solid-state memory device.2. The method of claim 1 where the removable solid-state memory devicecomprises flash memory.
 3. The method of claim 1 where the data beingavailable on the separate removable solid-state memory device is basedon an access frequency.
 4. The method of claim 1 where the data beingavailable on the separate removable solid-state memory device is basedon an access pattern.
 5. The method of claim 1 where the data beingavailable on the separate removable solid-state memory device is basedon a last access time.
 6. The method of claim 1 where the data beingavailable on the separate removable solid-state memory device is basedon a history.
 7. The method of claim 1 where the available datacorresponds to data stored on a sector of the hard disk.
 8. A computingdevice comprising: at least one processor; memory coupled to the atleast one processor; an external memory device manager implemented atleast in part by the at least one processor and the memory, and viawhich the computing device determines, in response to a read request fordata where the read request is directed to a hard disk, that the data isavailable from a removable solid-state memory device that is separatefrom the hard disk, where the separate removable solid-state memorydevice is configured to complete the read request faster than the harddisk; and the external memory device manager configured to redirect, inresponse to the computing device determining that the data is availablefrom the removable solid-state memory device, the read request from thehard disk to the separate removable solid-state memory device.
 9. Thecomputing device of claim 8 where the removable solid-state memorydevice comprises flash memory.
 10. The computing device of claim 8 wherethe data being available on the separate removable solid-state memorydevice is based on an access frequency.
 11. The computing device ofclaim 8 where the data being available on the separate removablesolid-state memory device is based on an access pattern.
 12. Thecomputing device of claim 8 where the data being available on theseparate removable solid-state memory device is based on a last accesstime.
 13. The computing device of claim 8 where the data being availableon the separate removable solid-state memory device is based on ahistory.
 14. The computing device of claim 8 where the available datacorresponds to data stored on a sector of the hard disk.
 15. At leastone computer storage media storing computer-executable instructionsthat, based on execution by a processor of a computing device, configurethe computing device to perform actions comprising: determining, by thecomputing device in response to a read request for data where the readrequest is directed to a hard disk, that the data is available from aremovable solid-state memory device that is separate from the hard disk,where the separate removable solid-state memory device is configured forcompleting the read request faster than the hard disk; and redirecting,by the computing device in response to the determining, the read requestfrom the hard disk to the separate removable solid-state memory device.16. The at least one computer storage media of claim 15 where theremovable solid-state memory device comprises flash memory.
 17. The atleast one computer storage media of claim 15 where the data beingavailable on the separate removable solid-state memory device is basedon an access frequency.
 18. The at least one computer storage media ofclaim 15 where the data being available on the separate removablesolid-state memory device is based on an access pattern.
 19. The atleast one computer storage media of claim 15 where the data beingavailable on the separate removable solid-state memory device is basedon a last access time, or where the data being available on the separateremovable solid-state memory device is based on a history.
 20. The atleast one computer storage media of claim 15 where the available datacorresponds to data stored on a sector of the hard disk.